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Evaluation figures as an important component in the 
Undergraduate Pre-Service Teacher Education Program (UPSTEP) at the 
University of Colorado. Its role is as follows: a) to help surface 
new instructional ideas that are latent in the experiences of staff 
members and articulate these ideas into program goals, b) to help 
shape program objectives based on the needs of teacher education, c) 
to assess the effectiveness of various component and general program 
operations, and d) to facilitate program changes based on the 
assessment of operations. . During the first year of the program, 
evaluation has been almost exclusively formative^ that is^ 
information has been gathered about the program while it is 
developing. However ^ evaluation demands an assessment of program 
outcomes^ and to this end the processes and product of the program 
have been assessed, . (The remainder of the paper is devoted to 
sections describing the process evaluation and product evaluation of 
UPSTEP.) (Related documents are SP 006 874^ 006 875^ and 006 867.) 
(JA) 
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The new tmdergraduate teacher training program at the Universi.ty 
of Colorado (UPSTEP) can be described as either an evaluators paradise 
or inferno. The potential contribution of evaluation in this program 
is practically unlimited. In fact, UPSTEP is more like five separate 
programs searching their particular identities in the context of more 
encompassing common goals. The architects of the program did envision 
a singular overriding framework. However, given the diversity and 
capability of its leadership, the implementation of this unitary concept 
has not proceeded unquestioned. The role of evaluation in the program 
therefore is to: 

1) Help surface new instructional ideas that exist latent 

in the experiences of staff members, and articulate these 
ideas into program goals% 

2) Help shape program objectives based on the needs of 
teacher education. 

3) Assess the effectiveness of various component and general 
program operations. 

4) Facilitate program changes based on the assessment of 
operations. 

Program evaluation during this first year has been almost 
exclusively formative ^ in contrast to siimmative . Formative evaluation 
refers to gathering information about the program while it is develop- 
ing, and discovering weaknesses and strengths. This provides a system- 
atic basis for program revision. Sunmiative evaluation concerns the 
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assessment of final program outcomes or finished products, Initial 
emphasis on formative evaluation clearly vras In order; the needs of 
the progrez: indicated that decisionmakers Ccomponent directors and 
staff) required information related to its development. This position 
is not one unique to DPSrSEP bi\t rather a necessary procedure of any 
new educational program. 

Although the major Gn^hasis has beer, on formative evaluation, 
sound evaluation demands an assessment of program outcomes. To this 
end, the evaluation procedure has assessed both processes — what is 
taking place in the program, and products — wha,t outcomes are being 
produced in the program. 

Process Evaluation 

The basic rationale for the use of evaluation in the program from 
the start is that sound decisions on the part of Its developers are 
necessary if the nev program Is to successfully accomplish its objectives. 
The process of developing an educational program such as UPSTEP involves 
a series of decision points that are critical to the pt^ogress and 
direction of the program. In order to make sound decisions, the developer 
must have knowledge of available alternatives. 

The various components of UPSTEP have continuously been at different 
stages of development. The objectives that are developed, the problems 
that arise, the questions that are asked and the decisions that have to 
be made^ differ from component to component. This situation demands 
that the evaluation proceed In a component by component fashion, but 
it also suggests that the program develop a procedure whereby the 
components can efficiently proceed toward development of program-wide 
aims and goals. 

During this first year, the Mathematics and Secondary Education 
components have been in the pre-operational stage of development. The 
first step for these two components was to devise a means of assessing 
the needs of teacher education in their respective areas, for example, 
the staff of the Math component is conducting a national survey of the 
needs, goals, and directions of math education programs. Similarly, 
^ the Secondary Education component surveyed Its constituents (former 
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.student^, public achool teacher?., graduate students and faculty mambers) 
l.'5 deternjine needs for change in secondary education. Out of this, 
the components began to formulate ohjectlves that vlll guide the 
development of thetr educational program* 

The Secondary Education component began the construction of a 
personalized, experiential, modular-based program. Based on the results 
of an experiment comparing tcethods of teaching, in addition to the 
aforementioned needs assessment, the Math component will design its 
courses for prospective teachers. In the process both components will 
be developing a formalized evaluation procedure to be used when their 
programs become operational. This procedure will be concerned with 
obcaiaing the information needed to determine if their program is 
progressing effectively and is accomplishing its specified outcome 
objectives. This work also will entail the development of instruments 
(or the search for existing evaltiative instruments) to obtain this infor-- 
mat ion . 

The operational components CScience, Social Science, and Elenieatary 
Education), during the fir^t year, needed to clarify the objectives 
under which they were operating* This process has helped to define 
the more encompassing goals of the entire program. That is, evaluation 
serves to identify aspects of the program held in common by the various 
components. 

The operational components th?n proceeded to develop evaluative 
questions to elicit inform^^ttion their staff requires to determine the 
effectiveness of the program. In turn, based on these questions, the 
components have designed and administered instruments eliciting 
appropriate information. Ttiis information then is being usei as 
"grist" for revisions and innovations m program operations. 

During the second year of operation it is planned that all 
components will staff their own evaluation monitoriag committees to 
carry out the development of evaluation plans, administration of 
evaluation instruments, and the interpretation and reporting of results. 
These monitoring committees will have the work of the previous year 
on xrfiich to build ^ basing the new measures on the needs of the individual 



component, Tha ioonitoring coimalttees will be composed qx at least one 
faculty member one TA ^xojn the coJijpQnent, plus the prggr^ evaluator. 
All component evnliiatiqn monitoring coimnittees vlll he rerponsihle to 
a central evaluation committee, Tbis coucnlttee is made up of faculty 
members and graduate students selected by the UPSTEP executive coranjittee 
and assigned tEie taak of providing conceptual leadership in evaluation 
and providing a model for conducting these activities in the program 
as a whole. 

Product Evaluation 

Product evaluation has proceeded more slowly than process 
evaluation^ The aspect cf progr^ evaluation involves, if not the 
more difficult task, the more patient ta^k of either selecting or 
developing instrumentation appropriate to the goals of the program. 
This instrmijentation prestnnes that functional program objectives have 
been clearly formulated. The first step toward sound product evalua-- 
tion is the development and clarification of the program's objectives; 
a process that has been an on-^going task for the first year operations. 

At least one component CSocial Science) has begun to gather data 
to be used in product evaluation. This component developed an instrument 
to test the student's understanding of economic concepts during the 
Fall semester. They currently are working on measures related to the 
knowledge and skills of political science, which is the content focus 
of the Spring semester. The component also has been studying the 
potential use of other instruments related to student attitudes towards 
teaching, student beliefs about teaching, and the conceptual behavior 
of prospective teachers, 

UPSTEP will be prepared to carry out a major product evaluation 
during the second year of operation. With program and individual 
component objectives constructed and clarified, subsequent search or 
development of outcome-related measures can proceed in an efficient 
manner. A main function of the program evaluator during this second 
year will be to oversee this product evaluation. The primary responsi^ 
bility for instrument construction, data gathering and data analysis 



will reat wltb tLie compaaent e.valuat.lon coEjmttteea, The program 
evaluator wtH guide this edtfcrt mid assure th& prggrain-vlde gut come 
goals are sufficiently assessed. 

Other Considerations 

It is also nec asar; for the evaluation to be concerned with those 
aspects of the program v^lct integrate the various components. For 
example, the effectiven . s of the Faculty/TA seminars has been a iajor 
consideration of evalu^^tion during the initial year. The shape anc 
direction of these seminars has had significant impact on the . ture of 
the program. During the second year the program avaluator will continue 
to monitor the progress of these «e«lnir8, in terms of botih process and 
product criteria. 

The evaluation process can be supplemented with related re ir- 
or evaluation conducted by program doctoral students fulfil] ^ng 
dissertation requirements. The eyaluator can identify progiani area.:. 
in need of study and assist the student *8 inquiry efforts into such 
problems . 

A final, but nonetheless strategic function of evalua .ion 11 
be to constantly appraise the needs of teacher education larg 
To this end the evaluator must revtev the literature on the subji : 
and ask, *'To what extent is the program addressing itself to trei is xn 
teacher f catxon?" Based on syntheses of the literature, it If chc 
function of the product evaluator to suggest revisions of program 
objectives which in turn can foster changes in program operations. 



